Section: New Results

Middleware for Grid and Cloud computing

Distributed algorithms for CAN-like P2P networks

Participants : Ludovic Henrio, Fabrice Huet, Justine Rochas.

The nature of some large-scale applications, such as content delivery systems or publish/subscribe systems, built on top of Structured Overlay Networks (SONs), demands application-level dissemination primitives which do not overwhelm the overlay, i.e. which are efficient, and which are also reliable. Building such communication primitives in a reliable manner would increase the confidence regarding their behavior prior to deploying them in real settings. In order to come up with real efficient primitives, we take advantage of the underlying geometric topology of the overlay network and we also model the way peers communicate with each other. Our objective is to design and prove an efficient (in terms of messages and execution time) and reliable broadcast algorithm for CAN-like P2P networks. To this aim, this year, we realized the following:

• publication in FASE 2013 of a formalisation, in Isabelle/HOL, of CAN-like P2P networks [15] . Thank to this work, we proved that there exist a broadcast algorithm that does not produce any duplicated message in those networks. A first naive algorithm was exhibited to prove it.

• design and and publication of an optimal broadcast algorithm for CAN-like P2P networks in OPODIS 2013 [20] . The solution we have proposed is proven to be correct, optimal in terms of number of messages, and also efficient, as it provides a good parallelization during the dissemination.

We are also investigating new algorithms to efficiently build a SON when the peer involved already have data. Most of the work on SONs assume that new peers joining the network will arrive without data and thus get assigned a random position. However, if they already have data, they will have to send them to other peers, depending on the key space they are responsible of. In 2013, we continued on the tracks investigated in 2012:

• We proposed a first version of new join algorithms which try to allocate key sub-spaces to peers so that the amount of data that needs to be moved is minimal. An expected benefit of this work is that it should allow for fast and efficient reconstruction of a SON in case of a crash, without having to use distributed snaphshots.

• We have conducted preliminary experiments which shows a reduction of data transfer between 20% and 90%.

Open Virtual Machines Placement Algorithms

Participants : Fabien Hermenier, Vincent Kherbache, Huynh Tu Dang.

Clients of IaaS providers are looking for dependable infrastructures that can cope with their SLA requirements. To stay attractive, a cloud must then rely on a Virtual Machine (VM) placement algorithm with features matching wrt the SLAs expectations. These constraints are however very specific to each of the tenants but also the infrastructure. They also cover a large range of concerns (reliability, performance, security, energy-efficiency, ...) that are continuously evolving according to new trends and new technologies. To address these issue, we advocate for a flexible VM placement algorithm that can be specialized through plugins to address new concerns.

This year, we first validate our approach with BtrPlace, a composable VM placement algorithm built over Constraint Programming [9] . The usage of Constraint Programming makes placement constraints independent of each other. New constraints can be added without changing the existing implementation. The expressivity of BtrPlace has been verified by implementing more than 20 placement constraints that reproduce, extend but also bring new meaningful restrictions on the VM placement with regards to constraints available in commercial placement algorithm. Each constraint was implemented by an average of 30 lines of Java code. An experienced developer implemented some of the them in half a day, while external developers, without any background in CP, have implemented constraints related to power efficiency  [43] .

Secondly, we exhibited a lack of reliability in the common approach to address placement constraints in some algorithms. Usually, a constraint controls the VM placement only at the end of the reconfiguration process and ignores the datacenter intermediary states between the beginning and the end of the reconfiguration process. In [11] , we advocated that this discrete approach is not sufficient to satisfy the SLAs continuously as an uncontrolled actions schedule may indeed lead to temporary violations. We relied on the flexibility provided by BtrPlace to exhibit these violations and to propose continuous constraints to control the quality of service at any moment. We implemented preliminary version of continuous constraints and confirmed they improve the datacenter reliability by removing any temporary violations.

GPU-based High Performance Cloud Computing

Participants : Michael Benguigui, Françoise Baude, Fabrice Huet.

To address HPC, GPU devices are now considered as unavoidable cheap, energy efficient, and very efficient alternative computing units. The barrier to handle such devices is the programming model: it is both very fine grained and synchronous. Our long term goal is to devise some generic solutions in order to incorporate GPU-specific code whenever relevant into a parallel and distributed computation. The first step towards this objective was to gain some insight on how to efficiently program a non trivial but well known algorithm. Our previous work  [40] highlights the necessity to target a GPU rather than distributed CPUs to provide the same performance level. By this way we price complex American basket options through the Picazo pricing algorithm, in the same order of time than a CPU cluster implementation on a 64-core cluster. This year, we achieved the following tasks

• We proposed a multi GPU based implementation of this method, allowing pricing time to fall below 1 hour on 18 GPUs, for a 40-assets American option [14] .

• We are currently designing a task dispatching model to load balance tasks in a CPU-GPU cluster. This will allow us to drastically lower the overall computation time of a portfolio estimation, and moreover, the computation time of the Monte Carlo value at risk of a portfolio of complex assets.

MapReduce Based Frameworks for Big Data

Participants : Fabrice Huet, Ge Song.

MapReduce is a programming model which allows the processing of vast amounts of data in parallel, on a large number of machines. It is particularly well suited to static or slow changing set of data since the execution time of a job is usually high. However, in practice data-centers collect data at fast rates which makes it very difficult to maintain up-to-date results. To address this challenge, we propose in [25] a generic mechanism for dealing with dynamic data in MapReduce frameworks. Long-standing MapReduce jobs, called continuous Jobs, are automatically re-executed to process new incoming data at a minimum cost. We present a simple and clean API which integrates nicely with the standard MapReduce model. Furthermore, we describe cHadoop, an implementation of our approach based on Hadoop which does not require modifications to the source code of the original framework. Thus, cHadoop can quickly be ported to any new version of Hadoop. We evaluate our proposal with two standard MapReduce applications (WordCount and WordCount-N-Count), and one real world application (RDF Query) on real datasets. Our evaluations on clusters ranging from 5 to 40 nodes demonstrate the benefit of our approach in terms of execution time and ease of use.

Another important point is the difficulty to predict the performance of a MapReduce job. This is particularly important when using pay-as-you-go resources such a Cloud. We have proposed a simple framework to predict the performance of Hadoop jobs. It is composed of a dynamic light-weight Hadoop job analyzer, and a prediction module using locally weighted regression methods. Our framework makes some theoretical cost models more practical, and also fits well with jobs and clusters diversity. It can also help those users who want to predict the cost when applying for an on- demand cloud service.